Multimedia streaming—the continuous delivery of synchronized media data like video, audio, text, and animation—is a critical link in the digital multimedia revolution. Today, streamed media is primarily about video and audio, but a richer, broader digital media era is emerging with a profound and growing impact on the Internet and digital broadcasting.
Locally synchronized media (or just “synchronized media”) means multiple media objects that share a common timeline. Video and audio are examples of synchronized media—each is a separate data stream with its own data structure, but the two data streams are played back in synchronization with each other. Virtually any media type can have a timeline. For example, an image object can change like an animated .gif file, text can change and move, and animation and digital effects can happen over time. This concept of synchronizing multiple media types is gaining greater meaning and currency with the emergence of more sophisticated media composition frameworks implied by MPEG-4, Dynamic HTML, and other media playback environments.
The term “streaming” is used to indicate that the data representing the various media types is provided over a network to a client computer on a real-time, as-needed basis, rather than being pre-delivered in its entirety before playback. Thus, the client computer renders streaming data as it is received from a network server, rather than waiting for an entire “file” to be delivered.
The same data can be simultaneously streamed to multiple client computers. In some situations, it is very important that this data be rendered by each of the multiple client computers at as close to the same time as possible (referred to as the data streams being “globally synchronized”). For example, live financial broadcasts (e.g., stock quotes or other reports) should be made available to all recipients at as close to the same time as possible. If one computer is rendering the streamed data before another, each of the data streams being streamed to the two computers can be referred to as being “out of global synchronization” or “globally unsynchronized”.
Rendering of data streams can become globally unsynchronized for a variety of different reasons. Bugs or other problems may exist in certain rendering devices (e.g., audio or video cards and/or their corresponding drivers) that cause the data on one computer to be rendered either slower or faster than expected. An example is clock-rate differences in the audio cards of rendering computers. This results in the data being rendered slower on one computer than on another, and the streams becoming more and more out of synchronization as rendering continues. Additionally, many modern computer systems are designed for the CPU to execute multiple different programs or processes concurrently (e.g., multi-tasking). In such a system, the CPU may spend a significant amount of time executing another process at the computer, causing the execution of the instructions for rendering the data stream at that computer to lag behind the execution of the instructions for rendering the data stream at another computer.
The invention described below addresses these disadvantages of data streaming, providing a way to resynchronize globally unsynchronized streams.